Learning Optical Flow, Depth, and Scene Flow Without Real-World Labels

Self-supervised monocular depth estimation enables robots to learn 3D perception from raw video streams. This scalable approach leverages projective geometry and ego-motion via view synthesis, assuming the world is mostly static. Dynamic scenes, which are common in autonomous driving human-robot interaction, violate this assumption. Therefore, they require modeling dynamic objects explicitly, for instance estimating pixel-wise motion, i.e. scene flow. However, simultaneous self-supervised learning of flow ill-posed, as there infinitely many combinations that result same point. In letter we propose DRAFT, a new method capable jointly depth, optical flow, by combining synthetic data with geometric self-supervision. Building upon RAFT architecture, an intermediate task bootstrap triangulation. Our algorithm also temporal consistency losses across tasks improve multi-task learning. DRAFT architecture simultaneously establishes state art all three setting on standard KITTI benchmark.